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Abstract—Objectives and information requirements are proposed for the design of a planning 
information system for Training and Personnel Technology RDT&E. Objectives are (1) 
improved procedures for relating Training and Personnel Technology RDT&E to defense 
objectives; (2) continuous coordination of plans through the establishment of a computer- 
assisted, DoD-wide RDT&E plans information sharing system; (3) improved labeling of work 
units and simplification of top management presentations by organizing large numbers of small 
technically-described work units into broad thrusts that identify deliverables with obvious 
payoff for defense; and (4) more systematic procedures for program evaluation and resource 
allocation decisions. Issues are identified that should be addressed during the design stage of 
such a system. 


The Department of Defense, like other organizations that conduct Research, Development, Test 
and Evaluation (RDT&E), must evaluate plans, develop investment strategies, and establish 
information systems in order to make decisions about the allocation of resources. In the past, 
these activities have tended to be unstandardized between Services and somewhat informal. In 
recent years, formal procedures for documenting and reviewing Training and Personnel 
Technology RDT&E efforts have been developed, and people-related RDT&E has been co- 
ordinated at higher levels within the Office of the Secretary of Defense (OSD). 


DIFFICULTIES IN MARKETING AND EVALUATING PLANS FOR 
TRAINING AND PERSONNEL TECHNOLOGY RDT&E 


Training and Personnel Technology RDT&E personnel often encounter difficulties in the 
marketing and evaluation of their plans. The plans are usually “softer” than hardware RDT&E 
plans, often take longer than expected, and don’t seem to “fit” with the broader needs of the 
Department of Defense as well as they should. Many people also think that behavioral 
scientists are not as skillful at marketing their plans as they should be. The development of a 
planning information system, consequently, requires some advance thought about problems and 
requirements. 

Part of the problem is that Training and Personnel Technology RDT&E really is different. 
For example: 

(1) Most people-related RDT&E is conducted in order to meet the broader needs of the 
Services (e.g. improved productivity or a better person-job-match system). These broader needs 
may or may not support specific hardware systems, which tend to dominate the RDT&E 
budgets of all the Services. It is difficult to incorporate people-related RDT&E into prioritiza- 
tion schemes that are primarily intended for decisions about hardware procurements. 


tThis paper was presented at the Seventh Defense Psychology Symposium which was held at the U.S. Air Force 
Academy in April, 1980. The views presented in this paper are those of the authors and do not necessarily reflect an official 
position of the U.S. Air Force. 
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(2) Support for people-related RDT&E has also been influenced by the fact that many of 
these efforts seem to benefit hundreds of activities without benefiting any one of them to a 
substantial degree. As a result, many recipients of these RDT&E products consider them ‘‘nice 
to have” rather than essential. Yet customer support is essential if people-related RDT&E is to 
have a beneficial impact on Service operations. 

(3) Another consideration is that most people-related RDT&E tends to be efficiency- 
oriented rather than combat-survival oriented. At a time when four-day wars are seriously 
contemplated as a possibility, efficiency-oriented RDT&E often seems less necessary than 
equipment-oriented expenditures, such as the purchase of another tank, aircraft, or missile. 
Some military analysts hold that it is better to have inefficient tank operators with more tanks 
than efficient operators with fewer tanks. There must be a function that states optimum 
relations, since it would be equally wrong at the limits to have expert operators with no tanks or 
many tanks with no trained operators. 


THE NEED FOR A NEW PLANNING INFORMATION SYSTEM 


A new planning information system could help resolve many current difficulties in the 
evaluation and marketing of plans for Training and Personnel Technology RDT&E. This is a good 
time to develop a system of this type. As noted by Taylor, many Congressional committees and 
their staffs have been concerned with how R&D products have been used to improve military 
capability. New planning systems and procedures might help alleviate these concerns. Another 
issue is duplication of effort due to a perceived lack of managerial and technical coordination and 
cooperation[1]. Information systems that can be shared by researchers and managers in many 
different parts of the country are now feasible with current computer capabilities. 


PREVIOUS WORK ON THE DESIGN OF RDT&E MANAGEMENT SYSTEMS 


Several reviews of the literature on RDT&E plans evaluation, project selection, and 
resource allocation have been published (e.g. BAKER and Pounp[2]; CeTron and Martino[3]; 
Aucoop([4}; CLARKE[5]; and SoupEr(6]). References of special interest are Nutt[7] and SCHOMAN, 
Dick and McNicut[8], both of which describe investment models specifically designed for 
Department of Defense (DoD) agencies. 

When Souder wrote his review in 1978, many years had elapsed since work on project 
selection models was initiated. From this historical perspective, Souder concludes that most 
project selection and evaluation models have not been useful because they were not designed to 
deal with real world environments. For example, the typical management science model seems 
to assume a single decision maker in a well behaved environment with accurate information. 
The real world, in contrast, usually involves many decision makers and many decision 
influencers in a dynamic organizational environment with information that is anything but 
perfect. Management science models also seem to assume that the goals of RDT&E are well 
known and invariant, whereas the real world usually involves ever changing, fuzzy goals, with 
the information needed to evaluate these goals scattered throughout the organization. People- 
related RDT&E organizations are also characterized by conflicting multiple objectives and 
multiple constraints, some of which are not economic in nature. In short, the real world is not 
as simple, logical, or quantifiable as management scientists would like it to be. 


MAJOR DESIGN OBJECTIVES 

Despite a complex situation with unusually difficult design problems, a DoD-wide, plans- 
oriented management information system for Training and Personnel Technology RDT&E is 
possible. The following design objectives for a system of this type are proposed: 

(1) Improved procedures for relating Training and Personnel Technology RDT&E to defense 
objectives. In spite of one admirable attempt to do so by the staff of the Assistant Secretary of 
Defense for Manpower, Reserve Affairs and Logistics[9], plans for people-related RDT&E 
have not been satisfactorily related to short and long range military defense objectives as seen 
by higher echelons. Some changes in the objectives or the way in which defense objectives are 
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seen by higher echelons may be needed. A plans evaluation system that can give more 
recognition to diffuse contributions of multipurpose Training and Personnel Technology 
RDT&E is also needed. 

(2) Continuous coordination of plans through the establishment of a computer-assisted, 
DoD-wide, RDT&E plans information sharing system. It is difficult to coordinate plans. They 
are always changing, and they are typically not very well defined. Dificult or not, however, 
co-ordination of plans in DoD is absolutely essential. The commanders of the various Training 
and Personnel Technology RDT&E organizations have been very active in this area, largely by 
scheduling multilevel coordination conferences [10]. More work is needed in this area, however, 
and additional work on interservice coordination is an important design consideration for a new 
system. 

A computer-assisted DoD-wide plans information sharing system seems the best way tu 
facilitate interservice coordination and reassure congressional committees and their staffs that 
there is no duplication of effort. Current systems of this type (e.g. Management and Scientific 
Information System (MASIS) and Defense Documentation Center searches) are largely limited 
to periodic checks of completed or on-going work. In contrast, a plans-oriented system could be 
used to provide a continuous check on future plans; to revise budgets for future years; to 
minimize planning paper work by providing a flexibly updated record system; and to store the 
input data needed for program evaluation and investment strategy decisions. 

(3) Improved labeling of work units and simplification of top management presentations. As 
noted by Dr. Ruth M. Davis, Deputy Under Secretary of Defense for Research and Engineer- 
ing, “‘In some quarters of the Armed Services Committee, there is a feeling that certain projects 
are a total waste and that nothing of operational value is coming from people-related R&D 
programs”’[10]. One reason for this lack of enthusiasm for people-related RDT&E is the way in 
which the multiplicity of RDT&E efforts is organized and labeled. 

There are more than a thousand people-related RDT&E efforts in the Department of 
Defense, and the titles usually specify technical objectives without any clear indication as to 
what the deliverable items may be. Higher level managers often look at titles without obtaining 
details or determining the contribution of the effort to defense objectives. They consequently 
tend to think that the many diverse efforts with technical titles are not making important 
contributions. If these hundreds of titles were organized into a few broad thrust headings that 
identify clear-cut products with obvious payoff for defense, communications would be greatly 
improved. It should be noted, however, that this requires changes in the way that RDT&E is 
managed—not just the way in which it is labeled. Small isolated RDT&E efforts that are not 
part of a larger thrust (and cannot be justified as basic research) must be scrutinized closely to 
be certain that they really deserve priority and do not have titles that will be seen as irrelevant 
for important defense objectives. The Air Force Human Resources Laboratory is currently 
implementing a thrust management approach of this type. 

(4) More systematic procedures for program evaluation and resource allocation decisions. 
Granted that most program evaluation and investment strategy decision methods make 
assumptions that do not correspond with the real world, there are still many techniques and 
procedures that could be useful. For example, a new planning information system for Training 
and Personnel Technology RDT&E should provide managers with information that would help 
them to reallocate resources if the Laboratory budget were to be substantially increased or 
decreased. A system could be designed to do this. Ideally, the information system would also 
consider the “missed opportunity costs” of each set of allocation decisions, by comparing the 
merits of efforts that are not funded with those that are. Any system that does not try to 
incorporate some sort of systematic resource allocation subsystem is not taking advantage of 
the state of the art. 


DESIGN ISSUES 
A number of systems design issues have been suggested in the literature. SicForp and 
ParviN[11] talk about 12 “mission concepts”; CETRoN, Martino and Roepcke(3] identify 15 
“features” of methods for evaluating and selecting R&D projects; KRASNICAN[12] identifies 12 
criteria for planning system evaluation; and CLARKE[S] identifies eight basic questions about the 
innovation decision process. Reference is also made to comments about information require- 
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ments attributed to Dr. Davis by Sanps{10] and Taytor[l]. Drawing generally from these 
references, the authors developed a list of issues that need to be dealt with during the design 
stage of a new system. The design issues are organized into five categories as follows: 

(1) How should plans be evaluated? Procedures are needed for: (a) deciding which factors 
should be considered at each decision point; (b) restating defense objectives so that they can be 
more easily translated into requirements for people-related RDT&E; (c) scoping proposals to a 
reasonable period of time and insuring adequate use of previously available data or knowledge; 
(d) identifying overlapping and redundant projects; (e) collecting evaluative information at 
specified points in time; (f) deciding if, when, and how mathematical models should be used for 
basic research, exploratory development, and advanced development; and (g) tracking possible 
changes in the nature of the problem and possible progress in closely related RDT&E. 

(2) How should resources be allocated? Plans are needed for: (a) deciding what kinds of 
resource allocation decisions should be made, who should make the decisions, and when they 
should be made; (b) rapidly reallocating funds when budget cuts take place or additional funds 
are made available on short notice; (c) determining the cost consequences of a change in priorities; 
and (d) considering ‘‘missed opportunity” costs. 

(3) What kind of an information system is needed? Detailed plans are needed for: (a) inputs, 
outputs and information flows; (b) the amount and kind of computer assistance to be provided 
at each point; (c) the way in which input data are coded, stored, processed, and retrieved; (d) 
output options; (e) storage requirements; (f) equipment requirements; (g) ways of identifying 
and weighting systems evaluation criteria; and (h) the costs of various options with respect to 
inputs, outputs, hardware configurations, operation and maintenance costs, and noneconomic 
considerations. 

(4) How could coordination be improved? Procedures are needed for: (a) analyzing inter- 
relationships among projects in order to decide how much coordination is needed; (b) docu- 
menting technical as well as managerial coordination; and (c) identifying the conditions under 
which organizational specialization (e.g. “‘lead agency” status) would be desirable. 

(5) What kinds of information should be provided to top management? The following 
options with respect to improved management information systems should be considered: (a) 
documenting RDT&E requirements by describing Service specific problems for which users or 
implementers are waiting for RDT&E results; (b) documenting past contributions by providing 
examples of how past products have been used or have provided improvements in capabilities; 
(c) illustrating expected contributions with easy to understand examples of policies or actions 
that could implement successful RDT&E; (d) grouping smaller efforts into large thrusts that 
have clearcut relevance for defense objectives; and (e) providing summary information about 
life cycle costs and projected benefits. 
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